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VISION 


KATHARINE  FRY 


WHICH  WAY  DO  WE  GO? 


ENTER 


CATEGORIES  OF  PROCESSING 


I  VP  =  Video  Processing 

Processing  <  IP  =  Image  Processing 

\  PP  =  Picture  Processing 


Classification 


PR  =  Pattern  Recognition 
II  =  Image  Interpretation 


I  SA  =  Scene  Analysis 

Understanding  /  IU  =  Image  Understanding 

CV  =  Computer  Vision 


A  HISTORY  OF  COMPUTER  VISION 


1950's 


EE 


Image  Processing  - 
enhancement, 
restoration,  coding, 
transmission 
Pattern  Recognition  - 
optical  character 
recognition 

BUT  -  no  understanding, 
mimics  human  vision 


A I 


Minimize  IP  by 
emphasizing  symbolic 
Drocessing 
Knowledge  directed 
vision 

Visual  knowledge 
representation 

BUT  -  Knowledge 
representation  is 
difficult  in  AI,  again  a 
computing  power 
shortage 


1960's 


CS 


Syntactic  Matching 
Recognition 
Understanding  3-D 
Scenes 

BUT  -  Insufficient 
knowledge  of  human 
vision,  insufficient 
computing  power, 
unreliable  feature 
detectors 


1980's 


Hierarchy  of  Representations 
Top-Down  and  Bottom-Up 


Computational  Neuroscience  Paradigm 


REPRESENTATIONAL  SCHEME  OF  MACHINE  VISION 


SCENE 


Scene  Features 
Perceptual  Attributes 


3-DMzinsic 
'  Images 


Labelling 

Naming 


Symbolic . 
Representation 


Sensing, 

Sampling, 

Quantization 

Primary 

Iconic 

Image 


Contrast  enhance¬ 
ment,  Noise  and 
Blurring 

Preprocessed 
Iconic  Image 


Primitive  Image 
Attributes: 
Reflectance,  Color, 
Texture,Motion, 
Edges 


Hypothesis 
Verbal  Scene 
Description 


Test 

Action 


Multisensory 

Integration 


COMPUTER  VISION  -  THE  BASIC  PROBLEM 


I 

1 


•  Information  is  lost  during  image  formation 

•  Inverse  mapping  from  images  to  scenes  is  not  unique 


t 


i 


Absorption 

Scatter 

Color 

Reflectance 

Intensity 

Spectrum  Of  Light 
Optics 

ImagingSystem 
Sensor  Transduction 


•  Image  understanding  is  difficult  because  we  do  not  understand  how  the 
various  confounding  processes  interact  in  the  resulting  image 


•  Image  understanding  is  difficult  because  we  do  not  understand  understanding 


THE  SEGMENTATION  PROBLEM 


Verbal  Representation 


Symbolic 

Domain 


Signal 

Processing 

Domain 


Pixel  Representation 


Segmentation 

-  partition  the  image  into  meaningful 
regions 


-  regions  are  2D  groupings  of 
elements  or  features  based  on: 
similarity,  proximity,  threshold 


Segmentation  is  difficult 

-  Conceptually 
Data  driven? 
Knowledge  driven? 
Both? 


-  Computationally  .  .  , 

Transition  in  the  data  structure  from  intensity  to  symbols 


VISION  MUST  BE  INTERDISCIPLINARY 


Provide 

Function  and  Mechanism  of 
vision 
perception 
cognition 
in  natural  systems 


COMPUTATIONAL  NEUROSCIENCE 


ASSUMPTIONS 


BRAIN  =  information  processsing  system 

known  properties  of  neurons  can  be  simulated 
with  available  technology 

information  processing  capabilities  of  neural  sy  stem 
do  not  involve  super-natural  principles  ( monism  -  dualism ) 


REDUCTIONIST  VIEW 


Essential  features  of  brain  functions  are  based  on: 
CONNECTIVITY 
SIGNAL  PROCESSING 
NETWORK  PLASTICITY 


COMPUTATIONAL  NEUROSCIENCE  VIEW 
OF  IMAGE  UNDERSTANDING 


REAL  TIME  REQUIREMENTS 


Perception  has  real  time  constraints 


Current  realizations  of  machine  perception  are  slow 


Need  New  Approach  - 

dedicated  parallel  architectures  and  distributed  computation 

Implication  - 

INSEPARABILITY  OF  STRUCTURE  AND  FUNCTION 


WHAT  ABOUT  PREVIOUS  WORK? 


•  DO  THEY  REALLY  WORK? 

•  HOW  WELL? 

•  IN  WHAT  DOMAINS? 

DON'T  KNOW - 

Because  to  evaluate  machine  vision  systems  is 
to  know  what  vision  is. 

CATCH  -  22 


OUR  APPROACH 


Human  Visual 
Task 

Performance 


Evaluating 
Performance 
of  Machine 
Vision 


Need  tasks  that  meaningfully  compare  human 
and  machine  vision 


GESTALT  CLOSURE  TEST 


•  Segmentation  task 

•  Incomplete  data  to  match  against  a  model 
What  is  the  model? 


How  much  data  is  needed? 


HIDDEN  FIGURES  TEST 


FIND 

THIS 

FIGURE 


IN  THIS  DRAWING 


-  Hard  for  people,  easy  for  machines 

-  Not  an  appropriate  visual  task  for  comparison 
comparison  and  evaluation 


’  V* 


SURFACE  TESTS 


Some  arc  easy: 


-  Rotation  and  manipulation  of  internal  models 

-  Easier  ones  can  be  solved  by  labelling  techniques 

-  Harder  ones  require  ? 


CONCLUSION 


-  New  approaches  needed  for  machine  vision 


-  How  do  you  evaluate  current  and  future  systems? 


-  Develop  evaluation  model  based  on  human 
visual  performance 


